arXiv:1502.07459vl [math.DS] 26 Feb 2015 


SHEARER’S INEQUALITY AND INFIMUM RULE FOR 
SHANNON ENTROPY AND TOPOLOGICAL ENTROPY 

TOMASZ DOWNAROWICZ, BARTOSZ FREJ AND PIERRE-PAUL ROMAGNOLI 


Abstract. We review subbadditivity properties of Shannon entropy, in par¬ 
ticular, from the Shearer’s inequality we derive the “infimum rule” for actions 
of amenable groups. We briefly discuss applicability of the “infimum formula” 
to actions of other groups. Then we pass to topological entropy of a cover. We 
prove Shearer’s inequality for disjoint covers and give counterexamples oth¬ 
erwise. We also prove that, for actions of amenable groups, the supremum 
over all open covers of the “infimum fomula” gives correct value of topological 
entropy. 


February 24, 2015 
1. Introduction 

This note is devoted to properties of entropy, both measure-theoretic and topo¬ 
logical, treated as a function defined on subsets of the acting group. One such 
property, called subadditivity, is popularly known and used. It implies, in particu¬ 
lar, that when evaluating the dynamical entropy (of a partition or of an open cover) 
in an action of Z, we can exchangeably apply lim sup, lim inf, lim or inf, of the terms 
^1?({ 1, 2,..., n}) (where H{{ 1, 2,..., n}) is appropriately understood), simply be¬ 
cause the sequence ^H({ 1, 2,..., n}) converges to its infimum. Similar statement 
holds for actions of countable amenable groups, with {1,2 ,,n} replaced by ele¬ 
ments F n of a Fplner sequence (and ^ replaced by jp-f )■ But the measure-theoretic 
entropy fulfills a stronger property, called strong subadditivity, which implies that 
the same value will be obtained when taking infimum of the terms over 

all finite subsets F of the acting group. Notice that this “infimum rule” allows to 
define (and evaluate) the entropy of a measure-preserving action of an amenable 
group without referring to any Fplner sequence. In other words, the simple form 
infi? jj^H(F) does not depend upon amenability of the group and can be used as a 
definition of entropy of processes in actions of arbitrary groups. A natural question 
arises: is this definition good (i.e., does it fulfill desirable postulates one expects 
from a reasonable notion of entropy)? We will briefly discuss this question in the 
section Beyond amenability. 

For the moment, it is not known whether a similar “infimum rule” applies to 
topological entropy of an open cover (even for the actions of Z). As one of our 
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examples shows, the corresponding function is not strongly subadditive, but this 
does not determine that the infimum rule fails. In fact, this rule is implied by 
a property lying between subadditivity and strong subadditivity, called Shearer’s 
inequality. We will show that if the open cover consists of disjoint sets, Shearer’s 
inequality holds, hence, for actions of countable amenable groups, the infimum 
rule does work. For non-disjoint covers, we show that Shearer’s inequality fails. 
Moreover, we and give an example of a Z 3 -action in which the infimum does not 
hold. We do not have an analogous example for Z—this open problem seems to be 
difficult. On the other hand, we prove that for amenable groups the supremum over 
all open covers of the “infimum formula” does yield the correct value of topological 
entropy of the action. 

We remark that all, presented in this note, results concerning measure-theoretic 
entropy are well known. Novel are only the results concerning topological entropy 
(in particular the examples). 

2. SUBADDITIVITY AND RELATED NOTIONS 

Let G be an abstract set and let F(G) be the collection of all nonempty finite 
subsets of G. A k-cover of a set F £ F(G) is a family {A'i, Kn ,..., K r } of elements 
of jF(G) (with possible repetitions) such that each element of F belongs to Ki for at 
least k indices i £ {1,2,..., r}. With slight abuse of precision, we will say “belongs 
to at least k elements of /C” and the sums and products over i £ {1,2,..., y} will 
be indexed by K £ 1C (we must remember that repeated terms K are counted 
separately). 

Let H be a nonnegative real function with domain T (G). 

Definition 2.1. We say that: 

(M) H is monotone if F C F' implies H(F) < H(F'); 

(S) H is subadditive if for any F,F' it holds that 

H(F U F') < H{F) + 

(Sh) FI satisfies Shearer’s inequality if for any F and any k-cover K. of F, 

H(F)<IJ2 H ^ 

Keic 

(SS) H is strongly subadditive if for any F,F' C G, 

H{F U F') < H(F) + H(F') - H(F n F'). 

Let us define the conditional value of FI by the formula 
(2.1) H{F\F’) = H(FU F') - H(F'). 

Using the conditional value we introduce two more notions: 

(MC) FI is monotone wrt. the condition if F' C F" implies H(F\F') > H(F\F"), 

(CS) FI is conditionally subadditive if for any F,F',F" it holds that 

H{F U F'\F") < H(F\F") + H(F'\F"). 

Remark 2.2. If we extend U by setting 11(0) = 0, then H{F |0) = H(F) for every 
F £ F(G) and monotonicity wrt. the condition includes that Fl{F\F') < F[(F). 

Lemma 2.3. If F[ is monotone then the conditions (SS), (MC) and (CS) are 
equivalent. 
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Proof. (SS) => (MC): For F' C F", 

H(F\F”) = H(F U F") - H(F") = H(F UF'U F") - H{F") 

< H(F U F') + H(F") - H((F U F') n F") - H(F") 

< H{F U F') - H{F') = H(F\F'). 

(MC) => (CS) (monotonicity is not used): 

H(F U F'\F") = H(F Uf'U F") - H{F") + H{F' U F") - H(F r U F") 

= H(F\F' U F") + H(F'\F") < H(F\F") + H(F'\F"). 

(CS) ==> (SS) (monotonicity is not used): 

H(F U F') = H(F U F' U (F n F')) = H(F U F'\F D F') + H(F n F') 

< H(F\F n F') + H(F'\F n F') + H(F n F') = 
H(F)~H(FnF , ) + H{F , )-H(FnF , ) + H(Fr\F') = H(F) + H(F')-H(FnF'). 


□ 

Proposition 2.4. If H is monotone then (SS) (Sh) => (S) and none of the 
implications may be reversed. 

Proof. Assume that H is strongly subadditive. Let F = {/i, / 2 , fm} and let /C 
be a k- cover of F. Using we can write 

and similarly, for each K £ 1C, 

H(K) = £ 

{rfj&K} 

By the preceding lemma, H fulfills (MC), hence H ({ f :j }|{ fi e K : i < j}) > 
H({fj}\{fi, ..., fj- 1 }). Since each fj belongs to at least k elements of /C, summing 
over /C, we obtain 

m 

H(K) > Y kHdfMh,..., fj- 1}) = kH(F). 

KZ )C j=1 

For the proof of (Sh) => (S) note that {F, F'\F} is a 1-cover of F U F', hence 
H(F U F') < H(F) + H(F' \F)< H(F ) + H{F'). 

For counterexamples we ask the reader to see section [6] In Example 16.41 we show 
that topological entropy of the standard time-zero partition (which is also a cover) 
in the golden mean shift is not strongly subadditive. On the other hand, it satisfies 
Shearer’s inequality, because the cover is disjoint (see Proposition 16.21) . 

In example 16.51 we present a ^-action such that topological entropy of a certain 
cover does not satisfy Shearer’s inequality (it does not even satisfy the infimum 
rule, see below). On the other hand, it is known that topological entropy of a cover 
is subadditive. □ 
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3. The infimum rule 


Let G be an amenable group and let (F n ) be a selected Fplner sequence. By |F| 
we will denote the cardinality of F. 


Definition 3.1. 

infimum rule if 


We will say that a nonnegative function H on F[G) satisfies the 


hmsiip f—H(F n ) = inf t^H(F). 

n—too \-Tn\ F^J r (G) \r \ 


Definition 3.2. We say that H is G-invariant if for any g G G it holds that 
H(Fg) = H(F). 


Proposition 3.3. If a nonnegative and G-invariant function H on F(G) satisfies 
Shearer’s inequality then it also obeys the infimum rule. 

Proof. Clearly, limsup^^ ^ H(F n ) > inf F6F(G) ^ H(F). 

For the converse inequality, fix an F £ F(G) and e > 0. For n large enough the 

1^? AF -1 F I 

Fplner set F n is (F _1 , e)-invariant, i.e., it satisfies 1 " | F | —^ < e. The family 

IC = {Fg : g£G,Fg^F n ^%} 

is a fc-cover of F n with k = |.F| (for g ^ g' , we treat the sets Kg and Kg' as 
different elements of the k- cover, even if they are equal as sets). Indeed, for / £ F n . 
the condition / £ Fg can be written as g £ F~ l f so, it is fulfilled for exactly k 
elements g. By the same calculation, Fg D F n ^ 0 if and only if g £ F~ 1 F n , so 
the cardinality of K, equals that of F~ 1 F n , i.e., it is not more than |.F n |(l + e). By 
invariance of H , H(K) = H(F) for every K £ 1C. The Shearer’s inequality now 
reads 

H{F n ) < j^\F n \(l + e)H(Ffi 

K Keic ' ' 

which, after dividing by |F ra | and passing with n to infinity, ends the proof. □ 


4. Shannon entropy 

Let (X, £, p) be a probability space. A partition of X is a finite collection V of 
pairwise disjoint measurable sets such that (J {eF A = X. By a join (or a common 
refinement ) of partitions V and V' we mean the partition V V V' = {A D B : A £ 
V, B £ V'}. Now, let G be an amenable group acting on X via measurable maps, 
which preserve the measure p. If F is a finite subset of G we write V F for the 
common refinement \/ geF g~ lr P, where g~ lr P = {g~ x A : A £ V} . Recall that the 
Shannon entropy of a partition V is defined by 

HrCP) = Y. p{A) log p(A) 

Aev 

and entropy of the action of G with respect to a partition V is defined as 
hn(G,V) = limsup ( V Fn ) 

n—¥ oo \t i n\ 

{{F n ) is a Fplner sequence in G). 

Proposition 4.1. The function H(F) = H^{V F ) is 
(1) nonnegative, 
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(2) G-invariant, 

(3) monotone, 

(4) strongly subadditive. 


These statements follow from standard properties of the entropy of a partition— 
for the proofs we refer the reader to any handbook on ergodic theory, ((SS) is 
usually replaced by (MC) or (CS), see e.g. [3] (1.6.7) and (1.6.9)]). In view of these 
facts, the following theorem follows from Propositions 12.41 and 13.31 


Theorem 4.2. Measure-theoretic entropy obeys the infimum rule, i.e. 

hp{G,V)= inf l-H^P*). 

fgf(G) |F| 

for every G-invariant measure p and every partition V. 


Remark 4.3. The above formula can be found, e.g. in [5], where it is attributed to 
Kolmogorov. 


5. Beyond amenability 

The “mindblowingly” simple formula inf Fe7 r( G ) jyjH g (V F ) can be applied to 
processes under actions of any countable groups. (It can be applied to uncountable 
groups as well, however, it will typically yield zero; such is the case of flows.) To 
distinguish from other existing notions, we will denote it by /i*(G, V). How good 
is this formula for countable non-amenable groups? The answer depends on the 
properties we expect from a good notion of dynamical entropy. 

The notion h*(G, V) has the following advantages: 

• It is completely universal, can be defined for arbitrary groups. 

• It is extremely simple, requires no details of the group (for instance in 
amenable groups it is formulated without referring to any Fplner sequence). 

• It satisfies the Bernoulli shifts postulate: Bernoulli shifts have “full” entropy 
(equal to the Shannon entropy of the independent generator). 

• It has a very convincing interpretation for other processes (entropy is lost 
in finite-dimensional dependencies and all such losses matter). 

Disadvantages can be detected by examining the action of the free group F 2 with 
two generators, and they include: 

• It fails the factors postulate: it can increase when passing to a factor. 

• It fails the invariance postulate: it can change with change of a generator 
(hence is not an isomorphism invariant). 

Before the examples, we recall the notions of the shift action and of a subshift. 
Suppose G is a group and A is a finite set with the discrete topology. By the full 
shift we understand the set A G (whose elements are x = (x g ) g ^G) equipped with 
the product topology, on which G acts by shifts: ( gx)h = Xh g - A subshift is any 
closed G-invariant subset X C A G . The full shift, as well as any subshift, admits a 
natural partition V\ = {[a] : a € A} by closed-and-open cylinder sets determined 
by the symbol “at zero”: [a] = {x : x e = a} (e is the unity of G). We call it 
the time-zero partition (or time-zero cover , depending on the context). The term 
Bernoulli measure is synonymous with a product measure z/ G on A G , where v is a 
probability measure on A. 
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Example 5.1. Let F 2 denote the free group with two generators a and b and unity 
e, and consider X = {—1, l }^ 1 2 with the shift action, the Bernoulli (^, ^)-measure, 
and the zero-coordinate partition V = {[— 1], [1]}. Clearly, H(V) = log 2 and 
h*^(F 2 , V) = log 2. Next, consider the mapping i/) : X —y {—1,1} x {—1,1} given by 

tjt(x) = (x(e)x(a), x(e)x(b)) 

and the associated four-element partition 7Z. It is not hard to see that the process 
generated by 1Z is the ^-Bernoulli shift: the one-dimensional distributions 

are independent. So, H^(1Z) = log4 and so equals h*(F 2 ,TZ). On the other hand, 
the process generated by 1Z is clearly a factor of that generated by V. Q 

Now let E = {e,a, b} C F 2 and consider Q = V E . Clearly, this partition 
is another generator of the process generated by V (the generated processes are 
isomorphic). For any finite set F C F 2 we have H(Q F ) = H{V EF ) = \EF\ log 2. 
However, the ratio does not drop below 2 (and can be arbitrarily close to 2). 
Hence /i*(F 2 , Q) = 2/i*(F 2 * * ,P) = log4. 0 

So, either we accept ft,* (G, V) as a parameter associated with a concrete process, 
maintaining its simplicity and interpretation, or we try to force it to become an 
isomorphism invariant. As an attempt in this direction we propose two invariants, 
both equal to ft*(G, P) for actions of amenable groups. Unfortunately, we are 
unable to verify whether these new notions fulfill the Bernoulli shift condition in a 
more general case. 

Definition 5.2. 


h**(X,G ) = inf{ft* (G, V) : V is a generator }, 

ft***(X, G) = : V is a generator}. 

Note that the latter notion has nothing to do with ft^G,? 7 ), we were driven 
to it just by analogy to ft**(X, G ). For actions of amenable groups we know that 
ft** ( X , G) equals ft M ( X , G). We also have ft*** ( X , G) = ft M ( X , G) (see (3 Corollary 
2.7]). 

In the general case it is obvious that h**(X, G) < h***(X, G). B. Seward [B] can 
prove the opposite inequality for free actions (i.e., such that for 3 / e, the set of 
points fixed by g has measure 0). As we already mentioned, we still do not know 
whether any of these notions satisfies the Bernoulli shifts postulate^ 


1 The above example shows that the failure of the factors postulate is inevitable for any entropy 
notion satisfying the Bernoulli shifts postulate (in particular, for sofic entropy my For this reason 
it is commonly agreed to give up the factors postulate in the search for a universal (i.e., valid for 
a range of acting groups) notion of entropy. 

2 The strength of the notion of sofic entropy is that it behaves better in this aspect; it does not 
depend on the partition as long as it generates the whole process. So, sofic entropy can be viewed 

as a parameter associated to measure-preserving actions, and becomes an isomorphism invariant. 

On the other hand, sofic entropy has its disadvantages: it has a very complicated definition, and 

it applies to actions of sofic groups only. 

^B. Weiss ([9]) can prove the Bernoulli shifts postulate for h***(X, G) in actions of sofic groups, 
so h***(X, G) becomes a serious competition for sofic entropy. It is unknown whether these two 

notions coincide for actions of sofic groups. 
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6. Topological entropy 

In the present section we enter the world of topological dynamical systems. We 
assume that X is a compact metric space and G acts by homeomorphisms on X. 
Similarly to the measure-theoretic case, for an open cover U and a finite FcMwe 
write U F for the refinement V/ge / _1 ^> where f~ x U = {f~ 1 U : U £ U}. We recall 
that N{U) is the smallest cardinality of a subcover chosen from a cover U and that 
topological entropy of a cover U is defined by H top (U) = log (N(U)). Topological 
entropy of the action is defined in two steps: 

hto P (T,U) = limsup —^—H top (U Fn ), 

n—> oo I-? n\ 

htop(T) = sup h top (T,U), 

u 

where ( F n ) is a Fplner sequence and the supremum is taken over all open covers 
of X. 

We want to study the nonnegative function on F[G) obtained by fixing an open 
cover U and abbreviating H top (U F ) as H top (F). It is obvious that H top is monotone 
and G-invariant. It is also commonly known (and easily verified) that this function 
is subadditive (see e.g. [5] (6.3.8)]). The natural next step is the verification of 
Shearer’s inequality. We begin with a discrete version of [1] Thm. 2], The proof is 
almost literally copied. 

Lemma 6.1. Let X be a subset of A", where A is a finite set and n £ N. Let 1C 
be a k-cover of the set of coordinates {1, 2,..., n} (the elements of 1C are nonempty 
subsets of {1, 2,..., n}, we admit repeated elements in 1C, and each coordinate be¬ 
longs to at least k elements of 1C). For K £ 1C let Xk denote the projection of X 
onto the coordinates belonging to K. Then 

i^i < n \% K \^- 

ife/c 

Proof For n = 1 the statement is obvious: each K £ 1C equals {1}, each Xk equals 
X and the cardinality of 1C is at least k. We proceed by induction. Consider an 
n > 2 and suppose the statement holds for subsets of A" -1 . The set X C A n splits 
into disjoint sets X a , a £ A, depending on the value at the last coordinate: 

X a = {(xi, x 2 , ■ ■ ■, x n ) £ X : x n = a}. 

For K £ 1C let K° = K \ {n} and let IC° = {K° : K £ 1C}. Clearly, 1C° is a 
k- cover of {1, 2,..., n — 1}. For every a £ A, X a can be viewed as a subset of A" -1 
(with the symbol a appended to each element), hence, by the inductive assumption, 
we have 

n &I*- 

KGK 

Because in X a the symbol at the last coordinate is determined, we have |X^- 0 | = 
|X^-| for every K, and we get 

i* a i < n 

ife/c 

|X| = ^|X°|, and \X k \ = J2 \XkI 
a£A A 


Further, 
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for every K £ K. such that n £ K. For K not containg n we will apply the estimate 
\X a K \ < \%k\ (regardless of a £ A). And so, we have 


(6.i) i*i=x;i*“i<x;ni^|i=£(n \x%\i- n m 1 *) 

aeA aGAKGlC aeA KBn KJn 

< n \% k \* • e n 

K^n a£A K3n 


For each K containing n, on A we define a function fx, by fxifl) 
then we apply the generalized Holder inequality: 


II fx 

K3n 


s n |a 


K3n 


\3Cx\i, and 


where - = i- Because K, is a k- cover, this sum has at least k terms, hence 

p < 1 (if p < 1, formally, || • || p is not a norm, but it does not matter). Since for a 
fixed finite-dimensional vector /, the term ||/|| p is a decreasing function of p > 0, 
the above inequality holds also for p = 1, and then it reads: 

e n i*£i*< n(EM'=n 

aGA K3n K3n aGA K3n 

Plugging this into EH) we end the proof. □ 


Corollary 6.2. IfU consists of pairwise disjoint sets then the corresponding func¬ 
tion H top on T(G) fulfills Shearer’s inequality. 

Proof. Because the cover U is by disjoint sets, for each F £ F{G) we have N{U F ) = 
\U F \, where U F is rid of empty elements. Labeling U by elements of a finite alphabet 
A (of the same cardinality asU) 1 U F can be identified with a set X C A F , while for 
any K C F we have U K = %k- Now, H top (F) = log |X| and H top (I\) = log \%k\ 
and the assertion follows directly from Lemma 16. II □ 

Corollary 6.3. If (X, G) is a subshift and U = Va then the infimum rule holds for 
Htop, i.e., 

h top (T) = h top (T,U) = inf l-H top {U F ). 

FGF(G) \t I 

Proof. Follows immediately from Propositions 16.21 and 13.31 □ 


The following example shows that topological entropy is not in general strongly 
subadditive, even for subshifts. To make matters worse, in another example we will 
present a system and a (non-disjoint) cover such that both Shearer’s inequality and 
infimum rule fail. 

Example 6.4. The golden mean shift is a subshift X C {0,1} Z consisting of all 
sequences in which block 11 does not occur. Let U = Va be the two-element time- 
zero cover. It was shown already that Shearer’s inequality holds, but it can easily 
be checked that 


#top({—1,0,1}) + Hto P ({0}) = log5 + log2 > log3 + log3 

= Ht op ({—l,0}) + ff tO p({0,l}), 


i.e., strong subadditivity does not hold. 
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The following example looks innocent, but it took us a lot of effort to find it. 

Example 6.5. Let A = {a, 6 , c}. We consider the action of the group Z 3 = Z/3Z = 
{0,1, 2}, by shifts, on a space X C A Za (i.e., a subshift) defined by 

X = {(a,a,a), (b,b,b), (c,c,c), (a,b,c), (b,c,a), (c,a,b)}. 

(In fact, (c, c, c) can be dropped and the example with 5 elements will still work.) It 
is clear that X is closed and Z 3 -invariant. The collection V = {{a, b}, { 6 , c}, {a, c}} 
is a (non-disjoint) cover of A. Let V = {V : V £ V}, where V = V x A x A. Clearly, 
V is a cover of A Zs hence also of X. The elements of V Z3 have the form V\ x V 2 x V 3 , 
where Vi, V 2 , V 3 are (not necessarily distinct) elements of V. It is easy to check that 
X admits a subcover of V Zs consisting of three sets, namely 

{a, b} x {a, b} x {a, b} : {a, c} x {£>, c} x {a, c}, { 6 , c} x {a, c} x {a, b}. 

On the other hand, there is no subcover with two elements (one set of the form 
Vi X V 2 x V 3 may contain at most two elements of X, because in any three of them, 
on some coordinate there appear all three letters). Thus N(V Z3 ) = 3 and 

fctop(Z 3 ,V) = p^topCP 23 ) = ilog3 

(in any finite group G, for any Fplner sequence, eventually F n = G ). However, 
the cover V^ 0,1 ^ has a minimal subcover consisting of only two sets (for example 
{ 6 , c} x {a, c} and {a, b} x {a, 6 }), and HtopO^ 0,1 }) = log 2. Since \ log 2 < i log 3, 
the inhmum rule does not hold. Consequently, by Proposition 13.31 (or by direct 
verification for the 2-cover of Z 3 by {0,1}, {1, 2}, {0, 2}), Shearer’s inequality fails 
as well. 

Remark 6 . 6 . Shearer’s inequality depends only vaguely on the acting group. The 
example (as a counterexample for Shearer’s inequality) can be easily adapted to the 
action of any group with at least 3 elements, in particular of Z. The infimum rule 
depends more heavily on the acting group (its proof uses only fc-covers obtained by 
shifting one set F of cardinality k), so the following question arises: 

Question 6.7. Let G be an infinite countable amenable group acting on a compact 
space X and let U be an open cover of X. Does the infimum rule hold for H top (F) = 
Hto P (bt F ), be., is it true that h top (G,U) = h* op {G,14)1 

In spite of many efforts, we have not succeeded in answering this questions even 
for G = Z. In fact, we do not even know toward which answer should we incline. 
Let us discuss the difficulties more extensively. 

Our example for Z 3 works only because the two-element set {0,1} and its shifts 
form a non-splitting 2-cover of Z 3 . In Z, an analogous 2-cover is splitting (i.e., it 
splits as a union of two 1 -covers) and it is easy to show, using plain subadditivity, 
that for a splitting k-cover of G, obtained by shifting one set F, the Shearer’s in¬ 
equality nearly holds (up to a small error) on large elements of the Fplner sequence. 
This suffices to prove the infimum rule. The simplest finite subset of Z whose shifts 
produce a non-splitting 3-cover is {0,1, 3}, but we failed to find a counterexample 
for Shearer’s inequality, based on any 3-cover. The point is that the key inequality 
i log k < log(fc + 1 ) does not hold for k > 2 . 

Nevertheless, we are able to formulate a positive result. In its proof we benfit 
from the fact that the infimum rule works for the Shannon entropy and we apply 
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the variational principle (for amenable groups it was first proved in [ 8 ]). Let us 
recall: if Mg(X) is the collection of all G-invariant probability measures on X 
then fttop(G) = sup^gj^^) h p (G). It suffices to take the supremum over ergodic 
measures. 


Theorem 6.8. Let X be a compact metric space and G a countable amenable group 
acting on X by continuous maps. Define 

h* top (G,U)=mi^-H top (U F ) 

h* op (G) = sup/i* (G,ZY) 
u 

Then 

K op (G) = h top (G) 

Proof. Clearly, h* op (G) < h top (G). 

To obtain the converse inequality, consider an ergodic measure /i on I, a finite 
partition V = {Ai 1 ..., A p } of X , and fix an e > 0. For any S > 0 one can choose 
compact sets B t C A i} i = 1 ,...,p, so that p(Ai \ Bi) < 8/p. Additionally, let 
B 0 = X \ U ? =1 Bi and define Ui = B 0 U Bi , i = 1, ...,p. Then U = {U\, ..., U p } is a 
cover with I/, D Ai, p(Ui \ Af) < 8. The family £ = {B 0 , B i,..., B p } is a partition 
of X. For any F G F(G), we have 

H P (V F ) < H p {V f V f F ) = H p (V F \f F ) + H,(t F ) < \F\H p (V\0 + H P ^ F ) 

(we have used subadditivity of the function H(F) = H p (/P F |£ F ), see e.g. 0 
(1.6.33)]. Choosing 8 appropriately small, we may demand that H p (V\f) < e, 
so that 


( 6 . 2 ) 


H p (P F )<H^ F ) + \F\e. 


Let ( F n ) be a tempered Fplner sequence (we skip the definition, every Fplner se¬ 
quence has a tempered subsequence, the ergodic theorem holds along any tempered 
Fplner sequence, see HI)- By the ergodic theorem, for almost all x it holds that 

lim 7177 l (9 S F n : gx G B 0 }\ = fi(B 0 ) < e. 

n->oo \p n \ 

The same is true for f~ 1 Bo (/ G F ), which implies that for almost all x, 
lim TTTT 1(5 G Fn ■ fgx S B 0 }\ < € 

n->oo \t n \ 

for all / G F. Therefore, we can choose N G N such that the set 

Xe.N = Pi n [x G X : —!— |{ ff G F n : fgx G B 0 }| < e 

n>N f£F ^ I n ' 

has positive measure. For x G X €> n and n > N we have 

f):f€F,g€F n , fgx G B 0 }\ < \F n \ • |F| • e, 


which implies 

(6.3) \{g G F n : \{f G F : fgx G B 0 }\ > |F| y/i'} \ < \F n \ y/l. 

Consider a collection of all sets of the form fl/eF / ^ C f'. where G/ = B 0 or 
Cf = X \ B 0: such that the first case occurs for at most |.F| yfe indices /. Let Y 
be the union of all such sets. Note that Y is a union of elements of the partition 
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£ f . The ratios p(-| \{g £ F n : gx £ Y}\ converge to g{Y) for //-almost every x. On 
the other hand, gx £ Y is equivalent to the fact that fgx £ Bq for at most |Fj y/e 
elements f £ F. If, in addition, we pick x £ X £j at then (16.31) implies that for every 
n> N, 

TT 7 T \{g £ F n : gx £ Y}\ > 1 - y/e. 

\-Tn\ 

Since X e ^ has positive measure, it contains a point which fulfills the ergodic the¬ 
orem for Y , implying that g(Y) > 1 — y/e. 

Let U' be a subcover of U F with minimal cardinality. Denoting by £ F the 
collection {B £ £ F : B D Y / 0} (note that we can as well write B C Y, so £ F is a 
partition of Y), we obviously have 

\&\ < E |{ Se ^ F :snt/^0}|. 

uew 

Fix some U £ U' , U = ("]/ 6 f 1 < jf < P- Consider a B £ £ F satisfying 

B fl U / 0. Then B is a cylinder in , B = f]j eF f~ 1 Bk f , 0 < kf < p, and 
because B is contained in Y, kf = 0 may occur for not more than |.F| y/e indices 
/ £ F. If kf / 0 then it must be equal to jf, because otherwise Bk f and Uj f would 
be disjoint, implying BnU = 0. Therefore, for given U the number of sets B e# 
intersecting U is estimated by the number of ways in which the (few) indices 0 can 
be distributed over the kf’ s, i.e., 

LICv^J 

{B £$: B<1U /:(/)} |< 

2—0 

It is well known that log (//) < nH(-), where H{5 ) is the binary entropy of the 
vector (5 ,1 — <5), so we get 

(6-4) log |$| < H top (U F ) + |F| H(y/e) + log \F\ V~e. 

We still need to compare the partition £ F with £ F . Let 1Z be the partition 
{Y, X \ Y} and let gy be the normalized conditional measure induced by g on Y. 
We have 

H^ F ) < H^ F vn)= H^e [R.) + H(K) 

= g(Y)H (^) + g(X \ Y)H^ y (£ f ) + H(H) 

<^v(^) + ^-log|e F |+H(^). 

Eventually, combining the above with (16.21) and (16.41) (and the standard estimate 
of entropy by means of cardinality), we get 

H„(V F ) < H top (U F ) + |F| H(yft + log \F\y/e+ y/e - log |£ F | + H(y/e) + \F\e. 

Dividing both sides by |F| and noticing that all but the first term on the right can 
be made arbitrarily small, regardless of F , by the choice of e, next taking infima 
over F on both sides, and supremum over IA on the right (which eliminates the 
small error terms), we arrive to 
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Now Theorem 14.21 allows to replace the left hand side by h ll (G : V). Since the 
inequality holds for any V and any ergodic /i, taking appropriate suprema (and 
applying the variational principle), we can further replace the left hand side by 
^top(G), concluding the proof. □ 
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